Autofocus control apparatus and method

ABSTRACT

Provided is an autofocus control apparatus in an image sensing apparatus which includes: an optical system including a focus lens; a solid-state image sensing device that photoelectrically converts light incident via the optical system into image signals and outputs the image signals; and a focus drive motor that drives the focus lens to adjust a focus position, the autofocus control apparatus comprising: a light-emitting section; and a focus position detector that performs focus position detection according to an active system and focus position detection according to a passive system on the basis of the image signals obtained from the image sensing unit.

FIELD OF THE INVENTION

The present invention relates to an autofocus control apparatus andmethod, and more specifically, to an autofocus control apparatus andmethod which perform focus control using image signals acquired by animage sensing device that photoelectrically converts an optical image ofan object formed by an image sensing optical system.

BACKGROUND OF THE INVENTION

Conventionally, there is an autofocus control apparatus that performsfocus control using image signals acquired by an image sensing elementthat photoelectrically converts an optical image of an object formed byan image sensing optical system.

As such an autofocus control apparatus, there has been proposed anautofocus control apparatus that uses both an autofocus system employingcontrast detection and an autofocus system employing infrared raydetection, and performs a focusing operation which uses the autofocussystem employing contrast detection at the time of a normal imagesensing operation and, on the other hand, switches to the autofocussystem employing infrared ray detection only under an image sensingenvironment in which the focusing operation according to the autofocussystem employing contrast detection is difficult, to thereby perform adistance measurement operation and an autofocus operation with respectto a desired subject. With such an apparatus, appropriate autofocus canbe performed regardless of brightness of a subject (e.g., see JapanesePatent Laid-Open No. 5-119250).

In addition, as another autofocus control apparatus, there has beenproposed an autofocus control apparatus that uses both an autofocussystem employing contrast detection, which photoelectrically converts anoptical image of an object to generate image signals and detects apredetermined high-frequency component from the generated image signalsto thereby perform focusing, and an autofocus system using alight-emitting unit (LED) irradiating an infrared ray and infrared raydetection, which receives reflected light from an object to detect anoutput signal corresponding to a distance to the object, and selects atleast one of the autofocus system employing contrast detection and theautofocus system employing infrared ray detection on the basis of anoutput from a temperature detection device which detects environmenttemperature to thereby perform an autofocus operation. According to thisautofocus control apparatus, appropriate autofocus can be performedregardless of a change in environment temperature when the apparatus isin use (e.g., see Japanese Patent Laid-Open No. 2000-111792).

However, in the above-described conventional examples, at least one ofthe autofocus system employing contrast detection and the autofocussystem employing infrared ray detection is selected according tobrightness of a subject or environment temperature when an apparatus isin use to perform an autofocus operation.

Consequently, there is a disadvantage that it is necessary to separatelyprovide an autofocus device employing infrared ray detection in additionto an image sensing device for photoelectrically converting an objectimage formed by an image sensing optical system, which leads to anincrease in cost. In addition, in the case in which one of the autofocussystems is selected to perform an autofocus operation, there is adisadvantage that a high-speed and high-precision autofocus operationcannot be performed.

SUMMARY OF THE INVENTION

The present invention has been devised in view of the above situation,and it is an object of the present invention to realize a highlyaccurate autofocus function at low cost.

According to the present invention, the foregoing object is attained byproviding an autofocus control apparatus in an image sensing apparatusthat comprises: an optical system including a focus lens; an imagesensing unit that photoelectrically converts light incident via theoptical system into image signals and outputs the image signals; and adrive unit that drives the focus lens to adjust a focus position, theautofocus control apparatus comprising: a floodlighting unit; and afocus position detector that performs focus position detection accordingto an active system and focus position detection according to a passivesystem on the basis of the image signals obtained from the image sensingunit.

According to the present invention, the foregoing object is alsoattained by providing an autofocus control method in an image sensingapparatus that comprises: an optical system including a focus lens; animage sensing unit that photoelectrically converts light incident viathe optical system into image signals and outputs the image signals; anda floodlighting unit, the autofocus control method comprising:performing focus position detection according to an active system on thebasis of the image signals obtained from the image sensing unit; andperforming focus position detection according to a passive system on thebasis, of the image signals obtained from the image sensing unit.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing a schematic structure of an imagesensing apparatus according to a first embodiment of the presentinvention;

FIG. 2 is a flowchart showing an image sensing operation of the imagesensing apparatus according to the first embodiment of the presentinvention;

FIG. 3 is a flowchart explaining processing of active AF according tothe first embodiment of the present invention;

FIG. 4 is a flowchart explaining object detection processing in theactive AF according to the first embodiment of the present invention;

FIG. 5 is a flowchart explaining the object detection processing in theactive AF according to the first embodiment of the present invention;

FIG. 6 is a diagram showing an example of arrangement of color filters;

FIG. 7 is a flowchart explaining processing of scan AF according to theembodiment of the present invention;

FIG. 8 is a diagram showing a relation between an amount ofhigh-frequency component and a focus lens position at the time when thescan AF is executed according to the embodiment of the presentinvention;

FIG. 9 is a flowchart explaining object detection processing in activeAF according to a second embodiment of the present invention;

FIG. 10 is a flowchart explaining object detection processing in theactive AF according to the second embodiment of the present invention;

FIG. 11 is a flowchart explaining object detection processing in activeAF according to a third embodiment of the present invention;

FIG. 12 is a flowchart showing a flow of an operation of active AFaccording to a fourth embodiment of the present invention;

FIG. 13 is a block diagram showing a schematic structure of an imagesensing apparatus according to a fifth embodiment of the presentinvention; and

FIG. 14 is a flowchart showing a flow of an operation of active AFaccording to the fifth embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described indetail in accordance with the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing an example of a schematic structure ofan image sensing apparatus according to a first embodiment of thepresent invention. The image sensing apparatus includes, for example, adigital still camera and a digital video camera but is not limited tothese cameras. It is possible to apply the present invention to anyimage sensing apparatus as long as the image sensing apparatus acquiresan incident optical image as an electrical image by photoelectricconversion using a solid-state image sensing device, such as an areasensor, whose pixels are arranged two-dimensionally.

In FIG. 1, reference numeral 1 denotes an image sensing apparatus; 2, agroup of zoom lenses; 3, a group of focus lenses; 4, a stop serving as alight amount adjusting unit and an exposure unit that controls an amountof light beams transmitted through an image sensing optical systemincluding the group of zoom lenses 2, the group of focus lenses 3, andthe like; 31, a lens-barrel including the group of zoom lenses 2, thegroup of focus lenses 3, the stop 4, and the like; 5, a solid stateimage sensing device such as a CCD (hereinafter referred to as “CCD”) onwhich an image of an object transmitted through the image sensingoptical system is formed and that photoelectrically converts thetransmitted image; 6, an image sensing circuit that receives electricsignals photoelectrically converted by the CCD 5 and applies variouskinds of image processing to the electric signals to thereby generateimage signals of a predetermined form; 7, an A/D conversion circuit thatconverts analog image signals generated by the image sensing circuit 6into digital image signals; 8, a memory such as a buffer memory (VRAM)that temporarily stores the digital image signals outputted from the A/Dconversion circuit 7; 9, a D/A conversion circuit that reads out imagesignals stored in the VRAM 8 to convert the image signals into analogsignals and converts the same into image signals of a form suitable forreproduction and output; 10, an image display device such as a liquidcrystal display device (LCD) (hereinafter referred to as LCD) thatdisplays the image signals; 12, a storage memory, which stores imagedata, comprising a semiconductor memory or the like; and 11, acompression/expansion circuit including a compression circuit, whichreads out the image signals temporarily stored in the VRAM 8 and appliescompression processing and coding processing of image data in order tochange the image signals to a form suitable for storage in the storagememory 12, and an expansion circuit, which applies decoding processingand expansion processing for changing the image data stored in thestorage memory 12 to a form optimal for performing reproduction,display, and the like of the image data.

In addition, reference numeral 13 denotes an AE processing circuit thatreceives an output from the A/D conversion circuit 7 and performsautomatic exposure (AE) processing; 14, a scan AF processing circuitthat receives an output from the A/D conversion circuit 7 and performsautofocus (AF) processing; 15, a CPU incorporating a memory forarithmetic operation for controlling the image sensing device; 16, atiming generator (hereinafter referred to as TG) that generates apredetermined timing signal; 17, a CCD driver; 21, a stop drive motorthat drives the stop 4; 18, a first motor drive circuit that controls todrive the stop drive motor 21; 22, a focus drive motor that drives thegroup of focus lenses 3; 19, a second motor drive circuit that controlsto drive the focus drive motor 22; 23, a zoom drive motor that drivesthe group of zoom lenses 2; and 20, a third motor drive circuit thatcontrols to drive the zoom drive motor 23.

Moreover, reference numeral 24 denotes an operation switch includingvarious groups of switches; 25, an EEPROM serving as an electricallywritable read-only memory in which a program for performing variouskinds of control, data to be used for causing the apparatus to performvarious operations are stored in advance; 26, a battery; 28, anelectronic flash; 27, a switching circuit that controls emission offlash of the electronic flash 28; 30, a light-emitting section forirradiating a light beam of infrared rays to an object including afloodlighting lens and a light-emitting device such as a light-emittingdiode; 32, an iR cut filter that blocks an infrared ray; and 33, afilter drive circuit for inserting and retracting the iR cut filter 32in front of the CCD 5.

Note that, as the storage memory serving as a storage medium for imagedata and the like, various forms of memories may be applied. Thoseinclude, inter alia, a stationary semiconductor memory such as a flashmemory, a semiconductor memory such as a card type flash memory that isformed in a card shape or a stick shape and is formed to be detachableattachable to the apparatus, and a magnetic storage medium such as ahard disk or a floppy disk.

As the operation switch 24, there are a main power supply switch forstarting the image sensing apparatus 1 and supplying power, a releaseswitch that starts an image sensing operation (storage operation), areproduction switch that starts a reproduction operation, a zoom switchthat instructs to move the group of zoom lenses 2 of the image sensingoptical system to perform zooming.

The release switch is constituted by two step switches including a firststroke (hereinafter referred to as SW1), which generates an instructionsignal for starting AE processing and AF processing to be performedprior to an image sensing operation, and a second stroke (hereinafterreferred to as SW2), which generates an instruction signal for startingan actual exposure operation.

Next, an operation of the image sensing apparatus in the firstembodiment having the above-described structure will be described.

First, a light beam from an object, which has been transmitted throughthe groups of lenses 2 and 3 in the lens-barrel 31 of the image sensingapparatus 1, is formed on a light-receiving surface of the CCD 5 after alight amount thereof is adjusted by the stop 4. This optical image ofthe object is converted into electric signals by photoelectricconversion processing by the CCD 5 to be outputted to the image sensingcircuit 6. The image sensing circuit 6 applies various kinds of signalprocessing to the inputted signals to generate image signals. Theseimage signals are outputted to the A/D conversion circuit 7 andconverted into digital signals (image data), and then temporarily storedin the VRAM 8.

The image data stored in the VRAM 8 is outputted to the D/A conversioncircuit 9, converted into analog signals, and converted into imagesignals of a form suitable for display, and is then displayed as animage on the LCD 10. On the other hand, the image data stored in theVRAM 8 is also outputted to the compression/expansion circuit 11. Aftersubjected to compression processing by the compression circuit in thecompression/expansion circuit 11, the image data is converted into imagedata of a form suitable for storage and stored in the storage memory 12.

In addition, for example, when a not-shown reproduction switch in theoperation switch 24 is turned on, a reproduction operation is started.Then, the image data, which is stored in the storage memory 12 in thecompressed form, is outputted to the compression/expansion circuit 11and subjected to decoding processing, expansion processing, or the likein the expansion circuit, and is then outputted to the VRAM 8 to betemporarily stored. Moreover, this image data is outputted to the D/Aconversion circuit 9 and converted to analog signals, and converted intoimage signals of a form suitable for display, and is then displayed asan image on the LCD 10.

On the other hand, the image data digitized by the A/D conversioncircuit 7 is also outputted to the AE processing circuit 13 and the scanAF processing circuit 14 when being outputted to the VRAM 8. In the AEprocessing circuit 13, in response to the inputted digital imagesignals, arithmetic operation processing such as cumulative addition andthe like is applied to a luminance value of image data of one frame.Consequently, an AE evaluation value corresponding to brightness of thesubject is calculated. This AE evaluation value is outputted to the CPU15.

In the scan AF processing circuit 14, in response to the inputteddigital image signals, a high-frequency component of image data of oneframe is extracted via a high-pass filter (HPF) or the like, andarithmetic operation processing such as cumulative addition or the likeis further performed. Consequently, an AF evaluation value correspondingto an amount of contour component on a high-frequency side iscalculated. In this way, the scan AF processing circuit 14 acts as apart of a high-frequency component detection unit which detects apredetermined high-frequency component from the image signals generatedby the CCD 5 in a course of performing the AF processing.

A predetermined timing signal is outputted to the CPU 15, the imagesensing circuit 6, and the CCD driver 17 from the TG 16, and the CPU 15performs various kinds of control in synchronization with this timingsignal. In addition, in response to the timing signal from the TG 16,the image sensing circuit 6 performs various kinds of image processingsuch as color signal separation or the like in synchronization with thetiming signal. Moreover, in response to the timing signal from the TG16, the CCD driver 17 drives the CCD 5 in synchronization with thetiming signal.

The CPU 15 controls the first motor drive circuit 18, the second motordrive circuit 19, and the third motor drive circuit 20, respectively, tothereby control to drive the stop 4, the group of focus lenses 3, andthe group of zoom lenses 2 via the stop drive motor 21, the focus drivemotor 22, and the zoom drive motor 23. In other words, the CPU 15controls the first motor drive circuit 18 on the basis of the AEevaluation value or the like calculated in the AE processing circuit 13to drive the stop drive motor 21 and performs AE control for adjustingthe stop 4 such that a stop value becomes appropriate. Further, the CPU15 performs AF control for controlling the second motor drive circuit 19on the basis of the AF evaluation value calculated by the scan AFprocessing circuit 14 and an output obtained by an active AF unit to bedescribed later to drive the focus drive motor 22 and move the group offocus lenses 3 to a focus position. In the case in which a not-shownzoom switch in the operation switch 24 is operated, in response to theoperation, the CPU 15 controls the third motor drive circuit 20 to drivethe zoom motor 23 to thereby move the group of zoom lenses 2 and performa magnification operation (zoom operation) of the image sensing opticalsystem.

Next, an image sensing operation of the image sensing apparatus 1 willbe described using a flowchart shown in FIG. 2.

When a main power switch of the image sensing apparatus 1 is ON and anoperation mode of the image sensing apparatus 1 is in an image sensing(recording) mode, an image sensing processing sequence is executed.

First, in step S1, as described above, the CPU 15 displays an opticalimage of an object, which is transmitted through the lens-barrel 31 andformed on the CCD 5, as an image on the LCD 10 via the image sensingcircuit 6, the A/D conversion circuit 7, the VRAM, and the D/Aconversion circuit 9.

Subsequently, in step S2, the CPU 15 confirms a state of the releaseswitch. When the release switch is operated by a user and the CPU 15confirms that the SW1 (first stroke of the release switch) is turned ON,the CPU 15 proceeds to the next step S3, and normal AE processing isexecuted. Then, in step S4, the CPU 15 performs active AF processing.The active AF processing is for detecting a rough distance to a desiredobject (focus position of the group of focus lenses 3) and is AFprocessing so-called coarse adjustment. The active AF processing will bedescribed in detail later.

After the active AF processing, in step S5, the CPU 15 performs scan AFprocessing (or passive AF) for fine tuning for detecting an accuratefocus position. In the scan AF processing, the CPU 15 performs the finetuning for detecting an accurate focus position while moving the groupof focus lenses 3 within a predetermined range around a found focusposition. Details of the scan AF processing will also be describedlater.

When the predetermined AF processing ends in this way, in step S6, theCPU 15 confirms the state of the SW2 (second stroke of the releaseswitch). If the SW2 is ON, the process proceeds to step S7 and the CPU15 executes actual exposure processing. Then, in step S8, the CPU 15stores an image obtained by exposure in the storage memory 12 via theCCD 5, the image sensing circuit 6, the A/D conversion circuit 7, theVRAM 8, and the compression/expansion circuit 11 as described above.

Next, details of the active AF processing executed in step S4 will bedescribed with reference to FIG. 3.

First, in step S20, the CPU 15 drives the filter drive circuit 33 toretract the iR cut filter 32 from the front of the CCD 5 so as not toattenuate an infrared ray component that is attenuated at the time ofimage sensing. This is because an infrared light-emitting device is usedas a light-emitting section 30 for the active AF processing sincereflectance of an infrared ray on various objects is more stable than anormal ray and good distance measurement can be expected.

Subsequently, in step S21, the CPU 15 performs charge accumulation withthe light-emitting section 30 on. The CPU 15 causes the light-emittingsection 30 to emit light to irradiate an infrared ray to the object. Atthe same time, the CPU 15 adjusts an amount of light to be received bythe CCD 5 with the stop 4. More specifically, the CPU 15 performscontrol such that the stop 4 is narrowed and an amount of light emissionof the light-emitting section 30 is increased when it is bright and thestop 4 is opened and the amount of light emission of the light-emittingsection 30 is reduced when it is dark. The incident optical image of theobject is converted into electric signals by the photoelectricconversion processing of the CCD 5 and outputted to the image sensingcircuit 6.

Next, in step S22, the CPU 15 controls the image sensing circuit 6 andreads out only a part of the output signals corresponding to a centralpart of the CCD 5 (hereinafter referred to as “detection area”). Then,in step S23, after outputting the read-out output signals to the A/Dconversion circuit 7 and converting it into digital signals (imagedata), the CPU 15 stores the digital signals in a memory incorporatedtherein. The digital signals are stored in the memory incorporated inthe CPU 15 because an amount of data is small, since only a part of theimage data from the CCD 5 has to be stored here. In addition, high-speedprocessing can also be realized by reading out only the part of theimage data in this way.

Then, the process proceeds to step S24 and the CPU 15 performs chargeaccumulation with the light-emitting section 30 off with the same stopas at the time of the charge accumulation with the light-emittingsection 30 on in step S21. The optical image of the object formed on thelight receiving surface of the CCD 5 is converted into electric signalsby the photoelectric conversion processing of CCD 5 and outputted to theimage sensing circuit 6 (step S25). The image sensing circuit 6 appliesvarious kinds of signal processing to the inputted signal to generateimage signals. The image signals are outputted to the A/D conversioncircuit 7 and converted into digital signals (image data), and are thentemporarily stored in the VRAM 8 (step S26). Then, the digital signalsare outputted to the D/A conversion circuit 9, converted into analogsignals, and converted into image signals of a form suitable fordisplay, and are then displayed as an image on the LCD 10.

The process proceeds to step S27 and the CPU 15 performs an externallight eliminating operation. Here, the CPU 15 finds differences betweenimage data of the charge accumulation with the light-emitting section 30on and image data of the charge accumulation with the light-emittingsection 30 off to thereby find data (object) of a reflected image thatis obtained as the infrared ray irradiated by the light-emitting section30 is reflected on the object. Since a component of an image formed byexternal light can be removed, it becomes easy to find a center of thereflected image. Practically, the CPU 15 reads data of a partcorresponding to the image data stored in the memory incorporated in theCPU 15 among the image data stored in the VRAM 8 and calculatesdifferences between the read image data and the image data stored in thememory incorporated in the CPU 15 to thereby remove external lightcomponents. An arithmetic operation result (differential data) with theexternal light eliminated in this way is stored in the memoryincorporated in the CPU 15.

Subsequently, in step S28, the CPU 15 calculates a center of gravity ofthe reflected image using the differential data calculated in step S27.Here, the CPU 15 extracts an object with a relatively high luminancehaving steep leading and trailing edges in the differential data andfinds a center of gravity of the extracted object.

Detection of an object is performed using signal values of one line thatare found by adding a plurality of lines of differential data. Notethat, in the case in which the surface of the CCD 5 is covered by acolor filter for color separation, only signal values corresponding to acolor filter with highest sensitivity to an infrared ray (red in aprimary color filter) are used. Consequently, since it becomesunnecessary to correct sensitivity of each color component of a colorfilter for an infrared ray as in the case in which all pixels are used,uncertainty due to sensitivity correction can be avoided and processingcan be performed at high speed.

Here, details of the processing for finding a center of gravity of anobject performed in step S28 will be described with reference to FIGS. 4and 5.

First, in step S400, the plurality of lines of differential data aremerged into one line.

Here, in the case in which a filter color arrangement for colorseparation covering the CCD 5 is a color arrangement as shown in FIG. 6,pixels corresponding to red are arranged in every other row and in everother pixel. Thus, the plurality of lines are merged into one line asdescribed below.

First, a differential data value of a j-th pixel of an L-th row isrepresented as a[L, j]. In addition, it is assumed that an i-th signalvalue of data for object detection arranged in one line is d[i]. In thecase in which a positional relation between a detection area of the CCD5 read in step S22 (area indicated by hatch lines in FIG. 6) and afilter arrangement are as shown in FIG. 6, since a filter correspondingto red exists in in odd number pixels of odd number rows, d[i] can berepresented as${d\lbrack i\rbrack} = {\sum\limits_{m}{{a\left\lbrack {{{2m} - 1},{{2i} - 1}} \right\rbrack}.}}$A range of m is from 1 to ½ of the number of rows in the detection area(decimals are raised to the next whole number). In other words, additionis performed from a first row (m=1) until the number of 2m−1 becomesequal to the number of the detection area (in the case in which thenumber of rows is an odd number) or to the number of rows of thedetection area−1 (in the case in which the number of rows is an evennumber). In addition, a range of i is from 1 to ½ of the number ofpixels in a horizontal direction of the detection area (decimals areraised to the next whole number). In other words, addition is performedfrom a first pixel (i=1) until 2i−1 becomes equal to the number ofpixels in the horizontal direction of the detection area (in the case inwhich the number of pixels is an odd number) or to the number of pixelsof the detection area−1 (in the case in which the number of pixels is aneven number).

For example, in the case in which seven rows form the detection area inthe active AF as shown in FIG. 6, a first pixel of added lines into asingle line is represented as follows:d[1]=a[1,1]+a[3,1]+a[5,1]+a[7,1].

In this way, data obtained by external light elimination stored in theincorporated memory is merged into data of a single line, and thedetection of an object is performed after storing the result of mergingthe data (hereinafter referred to as “line differential data”).

Note that the above arithmetic equations are applied to the case shownin FIG. 6 and is required to be changed appropriately according to anarrangement of color filters or a position of a detection area on theCCD 5.

Next, in step S401, the CPU 15 performs initialization of variables usedin an arithmetic operation and initialization of a work area of amemory. Here, the CPU 15 clears a work area in which counters i and kand data of an object are to be stored. In addition, since a pluralityof objects may be detected, a plurality of memory areas for anarithmetic operation are prepared in which data of the objects isstored.

Next, in step S402, the CPU 15 reads out the line differential datastored in the memory incorporated in the CPU 15 in order. Here, eachdata is represented as d[i]. Then, in step S403, the CPU 15 comparesd[i] with a first threshold value Th1. If d[i] is equal to or more thanthe first threshold value Th1, in step S404, the CPU 15 calculates adifference between d[i] and line differential data of a pixel that isaway from the pixel of d[i] by a plurality of pixels (e.g., two pixels),and checks if the difference is equal to or more than a second thresholdvalue Th2. If the difference is equal to or more than the secondthreshold value Th2, since it can be judged as a leading edge of anobject (i.e., a left end of the object), the process proceeds to stepS405 and various data of this object is stored in the memoryincorporated in the CPU 15. Consequently, an object with a relativelyhigh luminance having steep leading edge has been detected. On the otherhand, if the condition is not satisfied in step S403 or S404, theprocess proceeds to step S416 in FIG. 5.

In step S405, the CPU 15 finds a sum of luminance d[i] of signals and asum of products of luminance of signals and coordinates, d[i]×i, as dataof the object and stores the sums. In addition, the CPU 15 stores i asL[k] as the coordinate of the left end of the object. Now, when it isassumed that a sum of luminance of a k-th object is Sd[k] and a sum ofproducts of luminance and coordinates is Se[k], the following equationsare used.Sd[k]=d[i]+d[i−1]Se[k]=d[i]×i+d[i−1]×(i−1)

Next, in step S406, the CPU 15 increments the counter i by one to setthe next pixel as a subject of processing. In the next step S407, theCPU 15 judges if the processing has been finished for all the pixels ofthe line differential data. If the processing has been finished, theprocess proceeds to step S411, and if the processing has not beenfinished, the process proceeds to step S408. In step S408, the CPU 15reads out the next line differential data d[i]. Then, in step S409, theCPU 15 compares d[i] with the first threshold value Th1. If d[i] isequal to or more than the first threshold value Th1, the CPU 15 judgesthat the object still continues, and proceeds to step S410 and updatesthe data of the object in accordance with the following equations.Sd[k]=Sd[k]+d[i]Se[k]=Se[k]+d[i]×i

The CPU 15 performs these processes for updating data of the objectuntil the condition of step S409 are not satisfied while updating thecounter i in step S406. However, if the detection for all the pixels isfinished while these processes are continued (YES in step S407), sincethe condition of trailing edge is not satisfied, the CPU 15 judges thatthe object currently detected is inappropriate, clears the data of theobject in step S411, and the process immediately proceeds to step S418.

If the condition of step S409 is not satisfied (if d[i] is not equal toor more than the first threshold value Th1), since it can be judged thatthe object has ended, the CPU 15 checks whether or not the object has asteep trailing edge. In step S412, the CPU 15 calculates a differencebetween d[i] and line differential data of a pixel that is away from thepixel of d[i] by a plurality of pixels (e.g., two pixels), findsinclination at the trailing edge of the object, and checks if thedeclination is equal to or more than the second threshold value Th2. Ifthe declination at the trailing edge of the object is equal to or morethan the second threshold value Th2, since an object with a relativelyhigh luminance having steep leading and trailing edges has beenextracted, the process proceeds to step S413 and the CPU 15 updates thedata of the object in accordance with the following equations.Sd[k]=Sd[k]+d[i]Se[k]=Se[k]+d[i]×i

Moreover, the CPU 15 stores i in R[k] as the coordinate of the right endof the object. Then, the CPU 15 increments the counter k by one (stepS414) and updates an address of the memory storing the data of theobject. On the other hand, if the condition is not satisfied in stepS412, since an object with a relatively high luminance having steepleading and trailing edges has not been extracted, the CPU 15 clears thedata of the object that has been stored to that point (step S415).

Then, the process proceeds to step S416 and the CPU 15 increments thecounter i by one to set the next pixel as a subject of an arithmeticoperation, and then, in step S417, checks whether or not detection forall the pixels has been finished.

If the detection for all the pixels has been finished, in step S418, theCPU 15 finds a center of gravity of the detected object. The coordinatePx of the center of gravity of the object is calculated as follows.Px=Se[k]/Sd[k]

When a plurality of objects have been detected, the CPU 15 selectsobject/objects for which R[k]−L[k] is within a predetermined range. Thisis because, since a size of a reflected image of a light-emittingsection 30 on the CCD 5 should be substantially fixed, only reflectedimage/images falling within a fixed range anticipating an error is/areconsidered to be correct reflected image/images of the light-emittingsection 30. In the case in which a plurality of objects still remain, anobject for which a value of Sd[k]/(R[k]−L[k]) is the largest isselected. This is because, since the external light elimination has beenperformed for the object, it is considered that a luminance of areflected image of a light-emitting section 30 becomes the maximum.

After finding the center of the gravity of the reflected image in stepS28 of FIG. 3 as described above, in step S29, the CPU 15 finds adistance to the object to be sensed on the basis of a position of thecenter of gravity. A distance L to the object is found as follows basedon the principle of triangulation:L=(f+Δz)×B/x.Here, f is a focal length of an image sensing lens, Δz is an extensionamount of the image sensing lens, B is a baseline length (intervalbetween an optical axis of the light-emitting section 30 and an opticalaxis of the image sensing lens), and x is a deviation amount from acenter of an optical axis of the reflected image. Then, the CPU 15 findsa value indicating a position of the group of focus lenses 3corresponding to this distance L and sets it as a focus position G1 ofthe active AF.

In addition, if a center of gravity of the reflected image is not foundin step S28, since it is anticipated that the object is in the distanceand an intensity of the reflected image is small, the CPU 15 sets astarting point of a detection operation for a focus position by the scanAF processing to infinity and sets a focus position of the active AF asa value found by subtracting a variable Gs from infinity.

Note that it is also possible to repeat the processing from steps S21 toS29 plural times and set an average of the obtained focus positions as afocus position of the active AF.

Thereafter, the process proceeds to step S30 and the CPU 15 inserts theiR cut filter 32, which has been retracted from the front of the CCD 5in step S20, in front of the CCD 5 again.

Then, as described above, in step S5, the CPU 15 moves the group offocus lenses 3 to the vicinity of the focus position which has beenfound as a result of the distance measurement of the active AFprocessing.

Next, details of the scan AF executed in step S5 of FIG. 2 will bedescribed.

FIG. 7 is a flowchart showing processing of the scan AF. FIG. 8 shows arelation between an amount of high-frequency component and a focus lensposition at the time when the scan AF is executed.

In step S51 of FIG. 7, using the result of the arithmetic operation ofthe active AF processing, the CPU 15 sets a start position and a stopposition for driving the group of focus lenses 3 in order to perform adetection operation of a focus position by the scan AF processing. Avalue of the start position set here is found by subtracting a variableGs from the focus position obtained by the active AF (a value indicatingthe position of the group of focus lenses 3 corresponding to thedistance to the object as the result of the arithmetic operation of theactive AF processing) G1. In addition, a value of the stop position isfound by adding the variable Gs to the focus position G1 obtained by theactive AF. In other words, a range in which the scan AF is executed is apredetermined range around the position corresponding to the distance tothe object according to the result of the arithmetic operation of theactive AF processing. Note that the start position may be found byadding the variable Gs to G1 and the stop position may be found bysubtracting the variable Gs from G1. In which case, a driving directionof the group of focus lenses 3 is opposite to that in the case in whichthe values set in step S51 are adopted.

In addition, this variable Gs is determined taking into account a focallength of the image sensing lens, the result of the arithmetic operationof the active AF processing (distance to the object), parallax, adistance measurement error anticipated in the active AF processing, andthe like.

In general, a focusing operation in an image sensing apparatus is anoperation for focusing light beams from a desired object, which areconverged by an image sensing optical system, on an image sensingsurface (light-receiving surface) of an image sensing device (CCD,etc.). Therefore, the group of focus lenses 3, which are a part of theimage sensing optical system, are moved in an optical axis direction toobtain the focused state. An amount of the movement in the optical axisdirection tends to increase as the object is closer to the image sensingapparatus. In addition, the amount of the movement tends to increase asa focal length of the image sensing lens is longer.

Further, the parallax also tends to increase as the object is closer tothe image sensing apparatus and the focal length of the image sensinglens is longer.

Therefore, the variable Gs determining a range of the scan AF has alarger value as the distance to the object obtained from the result ofthe arithmetic operation of the active AF processing is closer and thefocal length of the image sensing lens is longer.

The distance measurement error includes, inter alia, an adjustment errorat the time of manufacturing the image sensing apparatus, a temperatureerror such as distortion caused by an environmental temperature changeof the lens-barrel 31, a distance measurement error due to a mechanicalerror of constituent members (the CCD 5, the light-emitting section 30,etc.) for performing the active AF, a movement error of the group offocus lenses 3, and the like are possible.

Therefore, the variable Gs is set to a value found from the distance tothe object obtained from the result of the arithmetic operation of theactive AF processing and the focal length of the image sensing lenstaking into account the distance measurement error.

For example, the variable Gs can be found by the following expression.Here, K1 is a constant, L is the distance to the object, f is the focallength of the image sensing lens, and δL is a coefficient for a distancemeasurement error or the like.Gs=K1×f/L+δL

Subsequently, in step S52, the CPU 15 drives the focus motor 22 via thesecond motor drive circuit 19 and moves the group of focus lenses 3 tothe start position set in step S51. Then, the CPU 15 executes the scanAF processing for finding a focus position while moving the group offocus lenses 3 with the start point as the origin at a predeterminedamount of movement.

In step S53, the CPU 15 judges whether or not the group of focus lenses3 have reached the end position set in step S51. If the group of focuslenses 3 have not reached the end position, the CPU 15 controls the CCD5 and the like to acquire image data corresponding to a position of thegroup of focus lenses 3 at that point. This image data is outputted tothe scan AF processing circuit 14 via the image sensing circuit 6 andthe A/D conversion circuit 7, and an AF evaluation value is calculatedin step S54. This AF evaluation value is outputted to the CPU 15 andstored in the memory for arithmetic operation incorporated in the CPU15. In the next step S55, the CPU 15 moves the group of focus lenses 3by a predetermined amount. Thereafter, the CPU 15 returns to step S53and repeats the same processing until the group of focus lenses 3reaches the set end position.

Then, when it is judged in step S53 that the group of focus lenses 3 hasreached the end position, the process proceeds to step S56. In step S56,the CPU 15 performs an arithmetic operation of a focus position on thebasis of the AF evaluation value calculated in step S54. Then, on thebasis of a result of the arithmetic operation, in step S57, the CPU 15drives the focus motor 22 via the second motor drive circuit 19 to movethe group of focus lenses 3 to the focus position. When the group offocus lenses 3 stops in this position, the CPU 15 ends the series ofsequences. Thereafter, the CPU 15 proceeds to step S6 of FIG. 2.

The series of operations in the san AF processing will be hereinafterdescribed with reference to FIG. 8.

For example, in a state in which the group of focus lenses 3 are in aposition indicated by A in FIG. 8, the group of focus lenses 3 firstmove from the position of A to a position B found by subtracting Gs froma result of distance measurement of the active AF that is the startposition (step S52). With this start position as the origin, the scan AFprocessing is executed until the group of focus lenses 3 reaches aposition C found by adding Gs to the result of distance measurement ofthe active AF that is the end position (steps S53 to S55). Then, on thebasis of an AF evaluation acquired by this, the CPU 15 performs anarithmetic operation for obtaining a focus position (step S56).According to this arithmetic operation, a position of D in FIG. 8, whichis a position of the group of focus lenses 3 corresponding to a peakvalue of a high-frequency component, is found as the focus position.Thereafter, the CPU 15 drives the group of focus lenses 3 to thatposition (step S57).

In this way, the CCD 5 is used as a light-receiving unit in the activeAF, a distance to the object is roughly calculated from outputs of aplurality of pixel columns thereof, and a position of the group of focuslenses 3, where a high-frequency component to be outputted is thelargest, is found from image signals generated by the CCD 5 while movingthe group of focus lenses 3 in a range set around a positioncorresponding to the roughly calculated distance, whereby the scan AFprocessing for detecting an accurate focus position is performed.Consequently, it is made possible to detect a focus position at highspeed and accurately with a low-cost structure simply added with thelight-emitting section for floodlighting an infrared ray.

Second Embodiment

Next, a second embodiment of the present invention will be described.

A basic constitution of an image sensing apparatus and its basicoperation procedure according to the second embodiment are the same asthe first embodiment. However, a method of detecting an object in theactive AF is different from the method that is described with referenceto FIGS. 4 and 5 in the first embodiment. Thus, the method will bedescribed.

In the active AF in the second embodiment, detection of an object isperformed in a plurality of rows, and a most appropriate object isselected out of obtained objects. In addition, as in the firstembodiment, only pixels corresponding to red of a color filter havinghighest sensitivity to an infrared ray are used.

Processing for extracting a reflected image and calculating a center ofgravity thereof, which is performed in step S28 of FIG. 3, will behereinafter described with reference to FIGS. 9 and 10. Here, an objectwith a relatively high luminance having steep leading and trailing edgeis extracted, and a center of gravity of the extracted object is found.

First, in step S800, the CPU 15 performs initialization of variablesused in an arithmetic operation and initialization of a work area of amemory. Here, the CPU 15 clears a work area in which counters i, j, andk and data of an object are to be stored. In addition, since a pluralityof objects may be detected, a plurality of memory areas for anarithmetic operation are prepared in which data of the objects isstored.

Next, in step S801, the CPU 15 reads out the differential data stored inthe memory incorporated in the CPU 15 in order. Here, each data isrepresented as d[i,j]. Then, in step S802, the CPU 15 compares d[i,j]with the first threshold value Th1. If d[i,j] is equal to or more thanthe first threshold value Th1, in step S803, the CPU 15 calculates adifference between d[i,j] and differential data of a pixel that is awayfrom the pixel of d[i,j] by a plurality of pixels (e.g., four pixels),and checks if the difference is equal to or more than a second thresholdvalue Th2. If the difference is equal to or more than the secondthreshold value Th2, since it can be judged as a leading edge of anobject (i.e., a left end of the object), the process proceeds to stepS804 and stores various data of this object in the memory incorporatedin the CPU 15. Consequently, an object with a relatively high luminancehaving steep leading edge has been detected. On the other hand, if thecondition is not satisfied in step S802 or S803, the process proceeds tostep S822 in FIG. 10.

In step S804, the CPU 15 finds a sum of luminance d[i,j] of signals anda sum of products of luminance of signals and coordinates, d[i,j]×i andd[i,j]×j, as data of the object and stores the sums. In addition, theCPU 15 stores j as L_(—)work as a coordinate of the left end of theobject. Now, when it is assumed that a sum of luminance of the object isSd_(—)work and a sum of products of luminance and coordinates isSe_(—)work, the following equations are used.Sd _(—)work=d[i,j]Se _(—)work=d[i,j]×i

In step S805, the CPU 15 increments the counter j by two to set adifferential data value of the next pixel corresponding red of a colorseparation filter as a subject of processing. In the next step S806, theCPU 15 judges if the value of the counter j indicates a coordinate of apixel outside the detection area. If the value indicates a coordinate ofa pixel outside the detection area, the process proceeds to step S824,and if the value indicates a coordinate of a pixel within the detectionarea, the process proceeds to step S807. In step S806, the CPU 15 readsout the differential data d[i,j] stored in the incorporated memory.Then, in step S808, the CPU 15 compares d[i,j] with the first thresholdvalue Th1. If d[i,j] is equal to or more than the first threshold valueTh1, since the object still continues, the process proceeds to step S809and updates the data of the object in accordance with the followingequations.Sd _(—)work=Sd _(—)work+d[i,j]Se _(—)work=Se _(—)work+d[i,j]×i

The CPU 15 performs these processes for updating data of the objectuntil the condition of step S808 are not satisfied while updating thecounter j in step S805. However, if the detection for all the pixels isfinished while thes processes are continued (YES in step S806), sincethe condition of trailing edge is not satisfied, the CPU 15 judges thatthe object currently detected is inappropriate, does not perform theprocessing for storing data of the object stored in the work area tothat point in the storage area, and the pocess immediately proceeds tostep S824.

If the condition of step S808 is not satisfied (if data is not equal toor more than the first threshold value Th1), since it can be judged thatthe object has ended, the CPU 15 checks whether or not the object has asteep trailing edge. In step S810, the CPU 15 calculates a differencebetween d[i,j] and differential data of a pixel that is away from thepixel of d[i,j] by a plurality of pixels (e.g., four pixels), findsinclination at the trailing edge of the object, and checks if theinclination is equal to or more than the second threshold value Th2. Ifthe declination at the trailing edge of the object is equal to or morethan the second threshold value Th2, since an object with a relativelyhigh luminance having steep leading and trailing edges has beenextracted, the CPU 15 stores j-2 as R_(—)work as a coordinate of theright end of the object in step S811.

Then, the CPU 15 proceeds to step S812 and performs processing forstoring the data of the object stored in the work area.

On the other hand, if the condition is not satisfied in step S810, sincean object with a relatively high luminance having steep leading andtrailing edges has not been extracted, the CPU 15 does not store thedata of the object stored in the work area to that point in the storagearea and the process proceeds to step S822.

In step S812 and subsequent steps, the CPU 15 performs processing forstoring the data of the object stored in the work area.

In this processing, first, the CPU 15 checks whether or not an object,which is the same as the object stored in the work area, already exists.

In step S812, the CPU 15 checks a value of the counter k indicating thenumber of areas in which the data of the object has been stored to thatpoint. If k is equal to zero, since the data of the object has not beenstored to that point, the CPU 15 proceeds to processing of step S819 andsubsequent steps for generating an area for a new object and storing thedata. This processing will be described later.

If k is not equal to zero, the CPU 15 initializes a value of the counterm to one in step S813 and sequentially checks the areas from the area inwhich the first object is stored. In step S814, the CPU 15 compares avalue L[m] of the left end of object data stored in an m-th storage areaand a value L_(—)work of the left end of object data stored in the workarea. If these values are substantially the same, the process proceedsto step S815. Subsequently, in step S815, the CPU 15 compares a valueR[m] of the right end of the object data stored in the m-th storage areaand a value R_(—)work of the right end of the object data stored in thework area. If the values are substantially the same, the processproceeds to step S816. In step S816, the CPU 15 checks if a value D[m]at the lower end of the object data stored in the m-th storage area andthe object of the work area are adjacent to each other in the verticaldirection. If the vertical direction coordinate i at the time when thedata of the work area is detected is D[m]+2, the value D[m] and theobject are adjacent to each other. In this case, since it can be judgedthat both of them are the same object, the process proceeds to step S817and the CPU 15 performs processing for storing the object data of thework area in the m-th storage area.

In step S817, the CPU 15 stores the data of the work area in the m-thstorage area as indicated below.Sd[m]=Sd[m]+Sd _(—)workSe[m]=Se[m]+Se _(—)workL [m]=Min(L[m],L _(—)work)R[m]=Max(R[m],R _(—)work)D[m]=i

Here, Sd[m] indicates a sum of luminance of signals, Se[m] indicates asum of products of luminance of signals and horizontal directioncoordinates, L[m] indicates the coordinate of the left end of theobject, R[m] indicates the coordinate o the right end of the object, andD[m] indicates the coordinate o the lower end of the object. Inaddition, Min(L[m],L_(—)work) is an arithmetic operation for selecting asmaller one of L[m] and L_(—)work, and Max(R[m], R_(—)work) is anarithmetic operation for selecting a larger one of R[m] and R_(—)work.Consequently, outer coordinates can be stored as coordinates at the leftand right ends. In addition, a suffix m in the brackets represents aparameter in the m-th storage area.

If the condition is not satisfied in steps S814, S815, or S816, in stepS818, the CPU 15 compares the counter m and the number of storage areask to thereby check if the comparison with all the existing storage areashas been finished. If m is equal to k, since the comparison with all thestorage areas has been finished, there is no object that is the same asthe object stored in the work area. Thus, the CPU 15 proceeds to stepS820 in order to generate a new storage area and store the object dataof the work area in the new storage area.

If the comparison with all the storage areas has not been finished (ifm<k), the CPU 15 increments the counter m by one in step S819, and thenreturns to step S814 and performs comparison with data of the nextstorage area.

In step S820, the CPU 15 increments a value of the counter k, whichindicates the number of areas storing data of the objects, by one toincrease the number of storage areas, and then proceeds to step S821,generates an area for storing data of a new object and stores the data.Storing in step S821 is performed as indicated below.Sd[k]=Sd_(—)workSe[k]=Se_(—)workL[k]=L_(—)workR[k]=R_(—)workU[k]=iD[k]=i

Here, Sd[k] indicates a sum of luminance of signals, Se[k] indicates asum of products of luminance of signals and horizontal directioncoordinates, L[k] indicates the coordinate of the left end of theobject, R[k] indicates the coordinate of the right end of the object,U[k] indicates the coordinate of the upper end of the object, and D[k]indicates the coordinate of the lower end of the object. In addition, asuffix k in the brackets represents a parameter in the k-th storagearea.

Then, the process proceeds to step S822 and increments the counter j bytwo to set the next pixel corresponding to red of the color separationfilter as a subject of an arithmetic operation, and then, in step S823,the CPU 15 checks whether or not processing for all the pixels in thehorizontal direction has been finished. If the processing for all thepixels in the horizontal direction has not been finished, the CPU 15returns to step S801 and continues the processing.

If the detection for all the pixels in the horizontal direction has beenfinished, in step S824, the CPU 15 increments the counter i by two toset the next row, on which a pixel corresponding to red of the colorseparation filter exists, as a subject of processing, and then, in stepS825, checks whether or not processing for all the rows has beenfinished. If the processing for all the rows in the vertical directionhas not been finished, the CPU 15 initializes j in step S826, and thenreturns to step S801 and continues the processing.

If the detection for all the rows has been finished, in step S827, theCPU 15 finds a center of gravity of the detected object. The coordinatePx of a center of gravity in the horizontal direction of the object isas indicated below.Px=Se[m]/Sd[m]In addition, since a center of gravity in the vertical direction ismeaningless in the case of finding a distance to the object in theactive AF processing, an arithmetic operation for finding the center ofgravity in the vertical direction is not performed.

Further, when a plurality of objects have been detected, anobject/objects for which both of R[m]−L[m] and D[m]−U[m] are within apredetermined range is/are selected. This is because, since a size of areflected image of a light-emitting section 30 on the surface of the CCD5 should be substantially fixed, only reflected image/images fallingwithin a fixed range anticipating an error is/are considered to becorrect reflected image/images of the light-emitting section 30. In thecase in which a plurality of objects still remain, first, a differencebetween (R[m]−L[m]) and (D[m]−U[m]) is checked, and an object with thesmallest difference is selected. This is because, in the case in which ashape of a light-emitting section 30 is directly reflected and focusedon the CCD 5, it is considered that sizes in the horizontal directionand the vertical direction thereof are substantially the same. However,since an error due to a form of an object to be sensed, a reflectivity,or the like occurs, object/objects having the difference/differenceswithin a predetermined range about the lowest difference is/areconsidered to have the substantially same lowest difference between(R[m]−L[m]) and (D[m]−U[m]). Then, an object with a largest value ofSd[m]/(R[m]−L[m]) is selected out of the objects. An object with alargest value of Sd[m]/(R[m]−L[m]) is selected because, since externallight elimination has been performed for the object, it is consideredthat a luminance of a reflected image of a light-emitting section 30becomes the maximum.

Third Embodiment

Next, a third embodiment of the present invention will be described.

A basis constitution of an image sensing apparatus and its basicoperation procedure according to the third embodiment are the same asthe first embodiment. However, a method of detecting an object in theactive AF is different from the method that is described with referenceto FIG. 4 in the first embodiment. Thus, the method will be described.

In the active AF in the third embodiment, detection of an object isperformed using differential data of one line, and a most appropriateobject is selected out of obtained objects.

Processing for extracting a reflected image and calculating a center ofgravity thereof, which is performed in step S28 of FIG. 3, will behereinafter described with reference to FIGS. 11 and 5. Here, an objectwith a relatively high luminance having steep leading and trailing edgesis extracted, and a center of gravity of the extracted object is found.

First, in step S500, the CPU 15 performs initialization of variablesused in an arithmetic operation and initialization of a work area of amemory. Here, the CPU 15 clears a work area in which counters i and kand data of an object are to be stored. In addition, since a pluralityof objects, which are considered to be a reflected image, may bedetected, a plurality of memory areas for an arithmetic operation areprepared in which data of the objects is stored.

Next, in step S501, the CPU 15 reads out differential data of apredetermined row stored in the memory incorporated in the CPU 15. Avalue of this data is represented as d[i]. Then, in step S502, the CPU15 compares d[i] with the first threshold value Th1. If d[i] is equal toor more than the first threshold value Th1, in step S503, the CPU 15calculates a difference between d[i] and differential data of a pixelthat is away from the pixel of d[i] by a plurality of pixels (e.g., twopixels), and checks if the difference is equal to or more than a secondthreshold value Th2. If the difference is equal to or more than thesecond threshold value Th2, since it can be judged as a leading edge ofan object (i.e., a left end of the object), the process proceeds to stepS504 and various data of this object is stored in the memoryincorporated in the CPU 15. Consequently, an object with a relativelyhigh luminance having steep leading edge has been detected. If thecondition is not satisfied in step S502 or S503, the process proceeds tostep S416 in FIG. 5.

In step S504, the CPU 15 finds a sum of luminance d[i] of signals and asum of products of luminance of signals and coordinates, d[i]×i, as dataof the object and stores the sums. In addition, the CPU 15 stores i asL[k] as the coordinate of the left end of the object. Now, when it isassumed that a sum of luminance of a k-th object is Sd[k] and a sum ofproducts of luminance and coordinates is Se[k], the following equationsare used.Sd[k]=d[i]+d[i−1]Se[k]=d[i]×i+d[i−1]×(i−1)

In step S505, the CPU 15 increments the counter i by one to set the nextpixel as a subject of processing. Then, in step S506, the CPU 15 judgesif the processing has been finished for all the pixels of thedifferential data of the current row. If the processing has beenfinished, the process proceeds to step S510, and if the processing hasnot been finished, the process proceeds to step S507. In step S507, theCPU 15 reads out the next differential data d[i] stored in theincorporated memory. Then, in step S508, the CPU 15 compares the d[i]with the first threshold value Th1. If d[i] is equal to or more than thefirst threshold value Th1, since the object still continues, the processproceeds to step S509 and updates the data of the object in accordancewith the following equations.Sd[k]=Sd[k]+d[i]Se[k]=Se[k]+d[i]×i

The CPU 15 performs these processes for updating data of the objectuntil the condition of step S508 are not satisfied while updating thecounter i in step S505. However, if the detection for all the pixels isfinished while these processes are performed (YES in step S506), sincethe condition of trailing is not satisfied, the CPU 15 judges that theobject currently detected is inappropriate, clears the data of theobject in step S510, and the process immediately proceeds to step S418of FIG. 5.

If the condition of step S508 is not satisfied (if d[i] is not equal toor more than the first threshold value Th1), since it can be judged thatthe object has ended, the CPU 15 checks whether or not the object has asteep trailing edge. As shown in FIG. 5, in step S412, the CPU 15calculates a difference between d[i] and differential data of a pixelthat is away from the pixel of d[i] by a plurality of pixels (e.g., twopixels), finds inclination at the trailing edge of the object, andchecks if the declination is equal to or more than the second thresholdvalue Th2. If the dclination at the trailing edge of the object is equalto or more than the second threshold value Th2, since an object with arelatively high luminance having steep leading and trailing edges hasbeen extracted, the process proceeds to step S413 and the CPU 15 updatesthe data of the object in accordance with the following equations.Sd[k]=Sd[k]+d[i]Se[k]=Se[k]+d[i]×i

Moreover, the CPU 15 stores i in R[k] as the coordinate on the right endof the object. Then, the CPU 15 increments the counter k by one (stepS414) and updates an address of the memory storing the data of theobject. On the other hand, if the condition is not satisfied in stepS412, since an object with a relatively high luminance having steepleading and trailing edges has not been extracted, the CPU 15 clears thedata of the object that has been stored to that point (step. S415).

Then, the process proceeds to step S416 and the CPU 15 increments thecounter i by one to set the next pixel as a subject of an arithmeticoperation, and then, in step S417, checks whether or not detection forall the pixels in the row has been finished.

If the detection for all the pixels in the row has been finished, instep S418, the CPU 15 finds a center of gravity of the detected object.The coordinate Px of the center of gravity of the object is calculatedas follows.Px=Se[k]/Sd[k]

It should be noted that, when a plurality of objects have been detected,the CPU 15 selects object/objects for which R[k]−L[k] is within apredetermined range. This is because, since a size of a reflected imageof a light-emitting section 30 on the CCD 5 should be substantiallyfixed, only reflected image/images falling within a fixed rangeanticipating an error is/are considered to be correct reflectedimage/images of the light-emitting section 30. In the case in which aplurality of objects still remain, an object for which a value ofSd[k]/(R[k]−L[k]) is the largest is selected. This is because, since theexternal light elimination has been performed for the object, it isconsidered that a luminance of a reflected image of a light-emittingsection 30 becomes the maximum.

Sensitivity to an infrared ray of each color of the color separationfilter arranged in front of the CCD 5 in detecting an object has beenneglected. However, since there is a difference of sensitivity inpractice, the processing for detection of an object is performed aftercorrecting respective pixel data in accordance with the sensitivity toan infrared light of each color of the filter, or as described in thefirst embodiment, only pixel data corresponding to red of a color filterhaving the highest sensitivity to an infrared ray are used.

In this way, the CPU 15 ends the processing for finding a center ofgravity of a reflected image in the third embodiment and returns to stepS28 of FIG. 3.

In addition, in the third embodiment, in performing the partial read-outprocessing in step S22 and storing the read-out partial image data instep S23 in the active AF processing shown in FIG. 3, an output of theA/D conversion circuit 7 is controlled to be inputted to the CPU 15.Thus, the output of the A/D conversion circuit 7 is not temporarilystored in the VRAM 8. Thus, an image due to charge accumulation with thelight-emitting section 30 on is not displayed on the LCD 10 and an imagetemporarily stored immediately before that is displayed.

An image due to charge accumulation with the light-emitting section 30on is not displayed in order to prevent an image causing a sense ofincongruity from being displayed on the LCD 10 due to the fact that onlya part of images are read out for speed-up of an arithmetic operationand a reflected image of a light-emitting device is imprinted. Inaddition, since an image due to charge accumulation with thelight-emitting section 30 off performed in step S24 of FIG. 3immediately after the charge accumulation with the light-emittingsection 30 on performed in step S21 of FIG. 3 is displayed, a sense ofincongruity such as a freezing screen is not given to a user.

Note that, in the third embodiment, since only differential data for oneline is used, it is advisable to control the partial reading performedin step S22 of the active AF processing shown in FIG. 3 so as to readout only a predetermined row used in the processing of FIG. 11 ratherthan reading out an area (detection area) in a central part as in thefirst embodiment.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described.

A basis constitution of an image sensing apparatus and its basicoperation procedure according to the fourth embodiment are the same asthe first embodiment. However, the fourth embodiment has acharacteristic in image display control at the time of the active AFprocessing.

In the fourth embodiment, unlike the third embodiment, image signals arestored in the VRAM 8 even at the time when a light beam is irradiated inthe active AF. In that case, a flag indicating that the light beam isirradiated is set such that the image signals stored in the VRAM 8 arenot outputted to the D/A conversion circuit 9 in a state in which thisflag is set. With such control, an image sensed at the time when a lightbeam is irradiated is not displayed on an image display apparatus.

Details of the display control will be hereinafter described withreference to FIG. 12.

First, in step S70, the CPU 15 drives the filter drive circuit 33 toretract the iR cut filter 32 from the front of the CCD 5 so as not toattenuate an infrared ray component that is attenuated at the time ofimage sensing. This is because an infrared light-emitting device is usedas a light-emitting section 30 for the active AF processing sincereflectance of an infrared ray on various objects is more stable than anormal ray and good distance measurement can be expected.

Subsequently, in step S71, the CPU 15 performs charge accumulation withthe light-emitting section 30 on. The CPU 15 causes the light-emittingsection 30 to emit light to irradiate an infrared ray to the object. Atthe same time, the CPU 15 adjusts an amount of light to be received bythe CCD 5 with the stop 4. More specifically, the CPU 15 performscontrol such that the stop 4 is narrowed and an amount of light emissionof the light-emitting section 30 is increased when it is bright and thestop 4 is opened and the amount of light emission of the light-emittingsection 30 is reduced when it is dark. Thereafter, in step S72, the CPU15 sets a flag for prohibiting output of image signals from the VRAM 8to the D/A conversion circuit 9.

Next, in step S73, a signal obtained by the charge accumulation with thelight-emitting section 30 on is stored in the VRAM 8 by hardware meansaccording to an instruction from the CPU 15. More specifically, areceived image of the object is converted into electric signals by thephotoelectric conversion processing of the CCD 5 and outputted to theimage sensing circuit 6. In the image sensing circuit 6, various kindsof signal processing are applied to the inputted signal to generateimage signals. These image signals are outputted to the A/D conversioncircuit 7 and converted into digital signals (image data), and are thentemporarily stored in the VRAM 8. However, the image data temporarilystored in the VRAM 8 is not outputted to the D/A conversion circuit 9.In this case, since the LCD 10 is driven in accordance with data whichwas set immediately before this image data is stored, the image due tothe charge accumulation with the light-emitting section 30 on is notdisplayed on the LCD 10, and an image acquired immediately before thatcontinues to be displayed.

Next, the process proceeds to step S74 and the CPU 15 performs chargeaccumulation with the light-emitting section 30 off with the same stopas at the time of the charge accumulation with the light-emittingsection 30 on in step S71. Simultaneously with this accumulation whichis, before outputting signals from the CCD 5 to the image sensingcircuit 6, the CPU 15 reads out only a part of the image data, whichcorresponds to a central part of the output signal from the CCD 5, fromthe VRAM 8 and stores it in the incorporated memory. This is because,since the CPU 15 only has to store a part, on which a reflected imagefrom a subject is formed, an amount of data is small. In addition,speed-up of processing can be expected by performing transfer of dataduring the charge accumulation with the light-emitting section 30 off.However, in the case in which the charge accumulation with thelight-emitting section 30 off ends before the transfer from the VRAM 8to the memory incorporated in the CPU 15 because the object to be sensedis extremely bright, the charge accumulation with the light-emittingsection 30 off is performed after the transfer from the VRAM 8 to thememory incorporated in the CPU 15 ends.

Next, in step S75, the signal obtained in the charge accumulation withthe light-emitting section 30 off is stored in the VRAM 8 by thehardware means according to an instruction from the CPU 15. After anamount of light from the object to be sensed is adjusted by the stop 4,the image formed on the light-receiving surface of the CCD 5 isconverted into electric signals by the photoelectric conversionprocessing by the CCD 5 to be outputted to the image sensing circuit 6.In the image sensing circuit 6, various kinds of signal processing areapplied to the inputted signals, and image signals of a predeterminedform are generated. These image signals are outputted to the A/Dconversion circuit 7 and converted into digital signals (image data),and are then temporarily stored in the VRAM 8.

Then, in step S76, the flag prohibiting transfer of image signals fromthe VRAM 8 to the D/A conversion circuit 9 is cleared. The image signalsread out from the VRAM 8 are converted into analog signals by the D/Aconversion circuit 9 and converted into image signals of a form suitablefor display, and are then displayed as an image on the LCD 10.

The process proceeds to step S77 and the CPU 15 performs an externallight eliminating operation. Here, the CPU 15 finds a difference betweenimage data of the charge accumulation with the light-emitting section 30on and image data of the charge accumulation with the light-emittingsection 30 off to thereby find data (object) of a reflected imageobtained as an infrared ray irradiated from the light-emitting section30 is reflected on the object. Since a component of an image formed byexternal light can be removed, it becomes easy to find a center of thereflected image. practically, the CPU 15 reads data of a partcorresponding to the image data stored in the memory incorporated in theCPU 15 among the image data stored in the VRAM 8 and calculatesdifferences between the read data and the image data stored in thememory incorporated in the CPU 15 to thereby perform external lightelimination. A result of the arithmetic operation (differential data)with the external light eliminated in this way is stored in the memoryincorporated in the CPU 15.

Subsequently, in step S78, the CPU 15 extracts a reflected image andcalculates a center of gravity thereof. The CPU 15 extracts an objectwith a relatively high luminance having steep leading and trailingedges, and finds a center of gravity of the extracted object. Since thisprocessing may be the processing described in any one of the first tothird embodiments, the description of the processing will be omitted.Then, in step S79, the CPU 15 finds a distance to the object from aposition of the center of gravity of the reflected image in the samemanner as the processing performed in step S29 of FIG. 3.

Thereafter, the process proceeds to step S80 and inserts the iR cutfilter 32, which was retracted from the front of the CCD 5 in step S70,in front of the CCD 5 again.

Fifth Embodiment

Next, a fifth embodiment of the present invention will be described.

In the fifth embodiment of the present invention, a VRAM 80, whichstores image signals obtained at the time when a light beam isirradiated in the active AF, is provided anew, and image signalsobtained at the time when a light beam is irradiated by thelight-emitting section 30 are stored in the VRAM 80. Since this VRAM 80is not connected to the D/A conversion circuit 9, an image sensed at thetime when a light beam is irradiated is not displayed on an imagedisplay apparatus.

A basic structure of an image sensing apparatus according to the fifthembodiment is shown in FIG. 13. The fifth embodiment is different fromthe first to fourth embodiments only in that the VRAM 80, which storesimage signals obtained at the time when a light beam is irradiated, isprovided anew. IA basic operation procedure is the same as the firstembodiment except that processing is performed using the VRAM 80 in theactive AF. An operation of the active AF in the fifth embodiment will behereinafter described with reference to FIG. 14.

First, in step S90, the CPU 15 drives the filter drive circuit 33 toretract the iR cut filter 32 from the front of the CCD 5 so as not toattenuate an infrared ray component that is attenuated at the time ofimage sensing. This is because an infrared light-emitting device is usedas a light-emitting section 30 for the active AF processing sincereflectance of an infrared ray on various objects is more stable than anormal ray and good distance measurement can be expected.

Subsequently, in step S91, the CPU 15 performs charge accumulation withthe light-emitting section 30 on. The CPU 15 causes the light-emittingsection 30 to emit light to irradiate an infrared ray to the object. Atthe same time, the CPU 15 adjusts an amount of light to be received bythe CCD 5 with the stop 4. More specifically, the CPU 15 performscontrol such that the stop 4 is narrowed and an amount of light emissionof the light-emitting section 30 is increased when it is bright and thestop 4 is opened and the amount of light emission of the light-emittingsection 30 is reduced when it is dark. The incident optical image of theobject is converted into electric signals by the photoelectricconversion processing of the CCD 5 and outputted to the image sensingcircuit 6.

Next, in step S92, the CPU 15 controls the image sensing circuit 6 toread out the electric signals only from a detection area correspondingto a central part of the output signal from the CCD 5. In the imagesensing circuit 6, various kinds of signal processing are applied to theinputted signals to generate image signals of a predetermined form.These image signals are outputted to the A/D conversion circuit 7 andconverted into digital signals (image data), and are then temporarilystored in the VRAM 80. Speed-up of processing can also be realized byreading only a part in this way. Since the VRAM 80 is not connected tothe D/A conversion circuit 9, an image obtained at the time of thecharge accumulation with the light-emitting section 30 on is notnaturally displayed on the LCD 10. An image, which has been displayedimmediately before this image data acquisition continues to be displayedon the LCD 10. With such display control, it is possible to prevent animage causing a sense of incongruity from being displayed on the LCD 10due to the fact that only a part of image signals are read out forspeed-up of an arithmetic operation and a reflected image of alight-emitting device is imprinted. In addition, since an image due tocharge accumulation with the light-emitting section 30 off performed instep S94 of FIG. 14 immediately after the charge accumulation with thelight-emitting section 30 on performed in step S91 is displayed, a senseof incongruity such as a freezing screen is not given to a user.

The process proceeds to step S93 and the CPU 15 performs the chargeaccumulation with the light-emitting section 30 off with the same stopas at the time of the charge accumulation with the light-emittingsection 30 on in step S21. The optical image of the object formed on thelight receiving surface of the CCD 5, after an amount of light thereofis adjusted by the stop 4, is converted into electric signals by thephotoelectric conversion processing of the CCD 5 and outputted to theimage sensing circuit 6. In the image sensing circuit 6, various kindsof signal processing are applied to the inputted signals to generateimage signals. These image signals are outputted to the A/D conversioncircuit 7 and converted into digital signals (image data), and are thentemporarily stored in the VRAM 8. Then, the image signals are outputtedto the D/A conversion circuit 9, converted into analog signals andconverted into image signals of a form suitable for display, and arethen displayed as an image on the LCD 10.

Next, the process proceeds to step S94 and performs an external lighteliminating operation. Here, the CPU 15 finds differences between imagedata of the charge accumulation with the light-emitting section 30 onand image data of the charge accumulation with the light-emittingsection 30 off to thereby find data of a reflected image obtained as alight beam irradiated from the light-emitting section 30 is reflected onthe subject. Since component of an image formed by external light can beremoved, it becomes easy to find a center of the reflected image.

Practically, the CPU 15 calculates differences between the image datastored in the VRAM 80 and the image data stored in the VRAM 8 to therebyperform external light elimination. More specifically, the CPU 15subtracts the image data, which corresponds to the data stored in theVRAM 80, stored in the VRAM 8 from the image data of the VRAM 80 andsequentially stores values the differences in the memory incorporated inthe CPU 15. In this way, transfer in terms of hardware with whichtransfer of data is performed at high speed is increased, and datatransfer in terms of software is reduced, whereby speed-up of anarithmetic operation can be expected.

Subsequently, in step S95, the CPU 15 extracts a reflected image andcalculates a center of gravity thereof. The CPU 15 extracts an objectwith a relatively high luminance having steep leading and trailing edgesand finds a center of gravity of the extracted object. Since thisprocessing may be the processing described in any one of the first tothird embodiments, the description of the processing will be omitted.Then, in step S96, the CPU 15 finds a distance to the object from aposition of the center of gravity of the reflected image in the samemanner as the processing performed in step S29 of FIG. 3.

Thereafter, the process proceeds to step S97 and the CPU 15 inserts theiR cut filter 32, which was retracted from the front of the CCD 5 instep S90, in front of the CCD 5 again.

The present invention is not limited to the above embodiments andvarious changes and modifications can be made within the spirit andscope of the present invention. Therefore to apprise the public of thescope of the present invention, the following claims are made.

1. An autofocus control apparatus in an image sensing apparatus thatcomprises: an optical system including a focus lens; an image sensingunit that photoelectrically converts light incident via said opticalsystem into image signals and outputs the image signals; and a driveunit that drives said focus lens to adjust a focus position, saidautofocus control apparatus comprising: a floodlighting unit; and afocus position detector that performs focus position detection accordingto an active system and focus position detection according to a passivesystem on the basis of the image signals obtained from said imagesensing unit; wherein said focus position detector, at the time of focusposition detection according to the active system, acquires first imagesignals with said image sensing unit performing floodlighting with saidfloodlighting unit, acquires second image signals with said imagesensing unit without performing the floodlighting with saidfloodlighting unit, and detects a focus position on the basis ofdifferential signals between the first image signals and the secondimage signals; wherein the first image signals are obtained from apredetermined partial area of said image sensing unit, and thedifferential signals are of the predetermined partial area; and whereinsaid focus position detector, prior to the detection of the focusposition, adds differential signals of the predetermined partial area ina predetermined direction to acquire one-dimensional added differentialsignals, and detects the focus position on the basis of the addeddifferential signals.
 2. An autofocus control apparatus in an imagesensing apparatus that comprises: an optical system including a focuslens; an image sensing unit that photoelectrically converts lightincident via said optical system into image signals and outputs theimage signals; and a drive unit that drives said focus lens to adjust afocus position, said autofocus control apparatus comprising: afloodlighting unit; and a focus position detector that performs focusposition detection according to an active system and focus positiondetection according to a passive system on the basis of the imagesignals obtained from said image sensing unit; wherein said focusposition detector, at the time of focus position detection according tothe active system, acquires first image signals with said image sensingunit performing floodlighting with said floodlighting unit, acquiressecond image signals with said image sensing unit without performing thefloodlighting with said floodlighting unit, and detects a focus positionon the basis of differential signals between the first image signals andthe second image signals; wherein said floodlighting unit irradiates aninfrared ray, and said image sensing unit is covered by a colorseparation filter, and image signals outputted from said image sensingunit are corrected to acquire the first image signals according tosensitivity of each color element of said color separation filter withrespect to an infrared ray.
 3. An autofocus control method in an imagesensing apparatus that comprises: an optical system including a focuslens; an image sensing unit that photoelectrically converts lightincident via said optical system into image signals and outputs theimage signals; and a floodlighting unit, said autofocus control methodcomprising: performing focus position detection according to an activesystem on the basis of the image signals obtained from said imagesensing unit; and performing focus position detection according to apassive system on the basis of the image signals obtained from saidimage sensing unit; wherein, in performing the focus position detectionaccording to the active system, acquiring first image signals with saidimage sensing unit performing floodlighting with said floodlightingunit, acquiring second image signals with said image sensing unitwithout performing the floodlighting with said floodlighting unit, anddetecting a focus position on the basis of differential signals betweenthe first image signals and the second image signals; wherein the firstimage signals are obtained from a predetermined partial area of saidimage sensing unit, and the differential signals are of thepredetermined partial area; and wherein, in performing the focusposition detection according to the active system, prior to thedetection of the focus position, adding differential signals of thepredetermined partial area in a predetermined direction to acquireone-dimensional added differential signals, and detecting the focusposition on the basis of the added differential signals.
 4. An autofocuscontrol method in an image sensing apparatus that comprises: an opticalsystem including a focus lens; an image sensing unit thatphotoelectrically converts light incident via said optical system intoimage signals and outputs the image signals; and a floodlighting unit,said autofocus control method comprising: performing focus positiondetection according to an active system on the basis of the imagesignals obtained from said image sensing unit; and performing focusposition detection according to a passive system on the basis of theimage signals obtained from said image sensing unit wherein, inperforming the focus position detection according to the active system,acquiring first image signals with said image sensing unit performingfloodlighting with said floodlighting unit, acquiring second imagesignals with said image sensing unit without performing thefloodlighting with said floodlighting unit, and detecting a focusposition on the basis of differential signals between the first imagesignals and the second image signals; wherein said floodlighting unitirradiates an infrared ray, and said image sensing unit is covered by acolor separation filter, said method further comprising acquiring thefirst image signals by correcting image signals outputted from saidimage sensing unit according to sensitivity of each color element ofsaid color separation filter with respect to an infrared ray.